System and method for enhanced streaming audio

ABSTRACT

A system and method for enhancement and management of streaming audio is disclosed. In one embodiment, the system provides a client-side decoder that is compatible with numerous audio formats, so that a user can enjoy relatively high-quality audio from various sources, even from sources that do not provide multi-channel or high-quality audio data. The system and method also include a management system for managing and controlling the use of licensed signal processing software to further enhance an audio stream. In one embodiment, the management system is used to manage a signal processing module that provides psychoacoustic audio processing to create a wider soundstage, an acoustic correction process to increase the perceived height and clarity of the audio image, and bass enhancement processing to create the perception of low bass from the small speakers or headphones typically used with multi-media systems and portable audio players.

REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.11/866,327, filed on Oct. 2, 2007, which is a continuation of U.S.application Ser. No. 09/734,475, filed on Dec. 11, 2000, titled “SYSTEMAND METHOD FOR ENHANCED STREAMING AUDIO,” now U.S. Pat. No. 7,277,767,the entirety of both of which is hereby incorporated by reference. Thepresent application also claims priority benefit of U.S. ProvisionalApplication No. 60/170,144, filed Dec. 10, 1999, titled “SURROUND SOUNDENHANCEMENT OF INTERNET AUDIO STREAMS,” and U.S. Provisional ApplicationNo. 60/170,143, filed Dec. 10, 1999, titled “CLIENT SIDE IMPLEMENTATIONAND MANAGEMENT TO INTERNET MUSIC AND VOICE STREAM ENHANCEMENT”, theentirety of both of which is hereby incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to techniques to enhance the quality ofstreaming audio, and techniques to manage such enhancements.

2. Description of the Related Art

Currently, streaming of audio via the Internet is beginning to overtakeradio in popularity as a method for distributing information andentertainment. At present, the formats used for Internet-baseddistribution of audio are limited to single-channel monaural andconventional two-channel stereo. Efficient transmission usually requiresthe audio signal to be highly compressed to accommodate the limitedbandwidth available. For this reason, the received audio is often ofmediocre or poor quality.

Due to bandwidth limitations, it is difficult to transmit more than twochannels of audio in real time via the Internet while maintaining audiointegrity. In order to effectively transmit more than two channels ofaudio over the Internet, multi-channel audio (typically meaning audiosources having two stereo channels plus one or more surround channels)must be encoded or otherwise represented by the two channels beingtransmitted. The two channels may then be converted into a data streamfor Internet delivery using one of many Internet compression schemes(e.g., mp3, etc). Systems that permit transmission of multi-channelaudio over traditional two-channel transmission media have significantlimitations, which make them unsuitable for Internet transmission ofencoded multi-channel audio. For example, systems such as DolbySurround/ProLogic are limited by: (i) their source compatibilityrequirements, making the audio delivery technique dependent upon aparticular encoding or decoding scheme; (ii) the number of channelsavailable in the multi-channel format that can be represented by the twochannels; and (iii) in the audio quality of the surround channels.Additionally, existing digital transmission and recording systems suchas DTS and AC3 require too much bandwidth to operate effectively in theInternet environment.

SUMMARY OF THE INVENTION

The present invention solves these and other problems by enhancing theentertainment value of Internet audio through the use of client-sidedecoders that are compatible with a wide variety of formats, enhancementof the audio stream (either client-side, server-side, or both), anddistribution and management of such enhancements.

In one embodiment, a Circle Surround decoder is used to decode audiostreams from an audio source. If a multi-channel speaker system (havingmore than two speakers) is available, then the decoded 5.1 sound can beprovided to the multi-channel speaker system. Alternatively, if a pairof stereo speakers is available, the decoded data can be provided to asecond signal-processing module for further processing. In oneembodiment, the second signal-processing module includes an SRSLaboratories “TruSurround” virtualization software module to allowmulti-channel sound to be produced by the stereo speakers. In oneembodiment, the second signal-processing module includes an SRSLaboratories “WOW” enhancement module to provide further soundenhancement.

In one embodiment, use of a licensed signal processing software module(the licensed software) is managed by a customized browser interface.The user can download the customized browser interface from a server(e.g., a “partner server”). The partner server is typically owned by alicensed entity that has obtained distribution rights to the licensedsoftware. The user downloads and installs the customized browserinterface on his or her personal computer. When playing a local audiosource (e.g., an audio file stored on the PC), the browser interfaceenables the licensed software so that the user can use the licensedsoftware to provide playback enhancements to the audio file. Whenplaying a remote file from an authorized server (i.e., from the partnerserver), the customized browser interface also enables the licensedsoftware. However, when playing a remote file from an unauthorizedserver (i.e., from a non-partner server), the customized browserinterface disables the licensed software. Thus, the customized browserinterface benefits the user by allowing enhanced audio playback. Thecustomized browser interface benefits the licensed entity by providedenhanced audio playback of audio streams from the servers managed orowned by the licensed entity. In one embodiment, the customized browserinterface includes trademarks or other logos of the licensed entity,and, optionally, the licensor. The authorized servers are servers thatare qualified (e.g., licensed, partnered, etc.) to provide the enhancedaudio service enabled by the customized browser interface.

One embodiment includes a signal processing technique that significantlyimproves the image size, bass performance and dynamics of an audiosystem, surrounding the listener with an engaging and powerfulrepresentation of the audio performance. The sound correction systemcorrects for the apparent placement of the loudspeakers, the imagecreated by the loudspeakers, and the low frequency response produced bythe loudspeakers. In one embodiment, the sound correction systemenhances spatial and frequency response characteristics of soundreproduced by two or more loudspeakers. The audio correction systemincludes an image correction module that corrects the listener-perceivedvertical image of the sound reproduced by the loudspeakers, a bassenhancement module that improves the listener-perceived bass response ofthe loudspeakers, and an image enhancement module that enhances thelistener-perceived horizontal image of the apparent sound stage.

In one embodiment, three processing techniques are used. Spatial cuesresponsible for positioning sound outside the boundaries of the speakerare equalized using Head Related Transfer Functions (HRTFs). These HRTFcorrection curves account for how the brain perceives the location ofsounds to the sides of a listener even when played back through speakersin front of the listener. As a result, the presentation of instrumentsand vocalists occur in their proper place, with the addition of indirectand reflected sounds all about the room. A second set of HRTF correctioncurves expands and elevates the apparent size of the stereo image, suchthat the sound stage takes on a scale of immense proportion compared tothe speaker locations. Finally, bass performance is enhanced through apsychoacoustic technique that restores the perception of low frequencyfundamental tones by dynamically augmenting harmonics that the speakercan more easily reproduce.

The corrected audio signal is enhanced to provide an expanded stereoimage. In accordance with one embodiment, stereo image enhancement of arelocated audio image takes into account acoustic principles of humanhearing to envelop the listener in a realistic sound stage. Inloudspeakers that do not reproduce certain low-frequency sounds, theinvention creates the illusion that the missing low-frequency sounds doexist. Thus, a listener perceives low frequencies, which are below thefrequencies the loudspeaker can actually accurately reproduce. Thisillusionary effect is accomplished by exploiting, in a unique manner,how the human auditory system processes sound.

One embodiment of the invention exploits how a listener mentallyperceives music or other sounds. The process of sound reproduction doesnot stop at the acoustic energy produced by the loudspeaker, butincludes the ears, auditory nerves, brain, and thought processes of thelistener. Hearing begins with the action of the ear and the auditorynerve system. The human ear may be regarded as a delicate translatingsystem that receives acoustical vibrations, converts these vibrationsinto nerve impulses, and ultimately into the “sensation” or perceptionof sound.

In addition, with one embodiment of the invention, the small pair ofloudspeakers usually used with personal computers can create a moreenjoyable perception of low-frequency sounds and the perception ofmulti-channel (e.g., 5.1) sound.

Further, in one embodiment, the illusion of low-frequency sounds createsa heightened listening experience that increases the realism of thesound. Thus, instead of the reproduction of the muddy or wobblylow-frequency sounds existing in many low-cost prior art systems, oneembodiment of the invention reproduces sounds that are perceived to bemore accurate and clear.

In one embodiment, creating the illusion of low-frequency soundsrequires less energy than actually reproducing the low-frequency sounds.Thus, systems, which operate on batteries, low-power environments, smallspeakers, multimedia speakers, headphones, and the like, can create theillusion of low-frequency sounds without consuming as much valuableenergy as systems which simply amplify or boost low-frequency sounds.

In one embodiment, the audio enhancement is provided by software runningon a personal computer, which implements the disclosed low-frequency andmulti-channel enhancement techniques.

One embodiment modifies the audio information that is common to twostereo channels in a manner different from energy that is not common tothe two channels. The audio information that is common to both inputsignals is referred to as the combined signal. In one embodiment, theenhancement system spectrally shapes the amplitude of the phase andfrequencies in the combined signal in order to reduce the clipping thatmay result from high-amplitude input signals without removing theperception that the audio information is in stereo.

As discussed in more detail below, one embodiment of the soundenhancement system spectrally shapes the combined signal with a varietyof filters to create an enhanced signal. By enhancing selected frequencybands within the combined signal, the embodiment provides a perceivedloudspeaker bandwidth that is wider than the actual loudspeakerbandwidth.

BRIEF DESCRIPTION OF THE DRAWINGS

The various novel features of the invention are illustrated in thefigures listed below and described in the detailed description thatfollows.

FIG. 1 is a block diagram showing compatible audio sources provided toaudio decoders and signal processors in a user's computer.

FIG. 2 is a block diagram showing interaction between a broadcast userand a broadcast partner.

FIG. 3 is a flowchart showing management of Internet audio streamenhancements.

FIG. 4 is a block diagram of a WOW signal processing system thatincludes a stereo image correction module operatively connected to astereo enhancement module and a bass enhancement system for creating arealistic stereo image from a pair of input stereo signals.

FIG. 5A is a graphical representation of a desired sound-pressure versusfrequency characteristic for an audio reproduction system.

FIG. 5B is a graphical representation of a sound-pressure versusfrequency characteristic corresponding to a first audio reproductionenvironment.

FIG. 5C is a graphical representation of a sound-pressure versusfrequency characteristic corresponding to a second audio reproductionenvironment.

FIG. 5D is a graphical representation of a sound-pressure versusfrequency characteristic corresponding to a third audio reproductionenvironment.

FIG. 6A is a graphical representation of the various levels of signalmodification provided by a low-frequency correction system in accordancewith one embodiment.

FIG. 6B is a graphical representation of the various levels of signalmodification provided by a high-frequency correction system for boostinghigh-frequency components of an audio signal in accordance with oneembodiment.

FIG. 6C is a graphical representation of the various levels of signalmodification provided by a high-frequency correction system forattenuating high-frequency components of an audio signal in accordancewith one embodiment.

FIG. 6D is a graphical representation of a composite energy-correctioncurve depicting the possible ranges of sound-pressure correction forrelocating a stereo image.

FIG. 7 is a graphical representation of various levels of equalizationapplied to an audio difference signal to achieve varying amounts ofstereo image enhancement.

FIG. 8A is a diagram depicting the perceived and actual origins ofsounds heard by a listener from loudspeakers placed at a first location.

FIG. 8B is a diagram depicting the perceived and actual origins ofsounds heard by a listener from loudspeakers placed at a secondlocation.

FIG. 9 is a plot of the frequency response of a typical smallloudspeaker system.

FIG. 10 is a schematic block diagram of an energy-correction systemoperatively connected to a stereo image enhancement system for creatinga realistic stereo image from a pair of input stereo signals.

FIG. 11 is a time-domain plot showing the time-amplitude response of thepunch system.

FIG. 12 is a time-domain plot showing the signal and envelope portionsof a typical bass note played by an instrument, wherein the envelopeshows attack, decay, sustain and release portions.

FIG. 13 is a signal processing block diagram of a system that providesbass enhancement using a peak compressor and a bass punch system.

FIG. 14 is a time-domain plot showing the effect of the peak compressoron an envelope with a fast attack.

FIG. 15 is a conceptual block diagram of a stereo image (differentialperspective) correction system.

FIG. 16 illustrates a graphical representation of the common-mode gainof the differential perspective correction system.

FIG. 17 is a graphical representation of the overall differential signalequalization curve of the differential perspective correction system.

In the figures, the first digit of any three-digit number generallyindicates the number of the figure in which the element first appears.Where four-digit reference numbers are used, the first two digitsindicate the figure number.

DETAILED DESCRIPTION

FIG. 1 is a block diagram showing an audio delivery system 100 thatovercomes the limitations of the prior art and provides a flexiblemethod for streaming an encoded multi-channel audio format over theInternet. In FIG. 1, one or more audio sources 101 are provided,typically through a communication network 102, to a computer 103operated by a listener 148. The computer 103 receives the audio data,decodes the data if necessary, and provides the audio data to one ormore loudspeakers, such as, loudspeakers 146, 147, or to a multi-channelloudspeaker system (not shown). The audio sources 101 can include, forexample, a Circle Surround 5.1 encoded source 110, a Dolby Surroundencoded source 111, a conventional two-channel stereo source 112(encoded as raw audio, MP3 audio, RealAudio, WMA audio, etc.), and/or asingle-channel monaural source 113. In one embodiment, the computer 103includes a decoder 104 for Circle Surround 5.1, and, optionally, anenhanced signal processing module 105 (e.g., an SRS LaboratoriesTruSurround system and/or an SRS Laboratories WOW system as described inconnection with FIGS. 4-17). The signal processing module 105 is usefulfor a wide variety of systems. In particular, the signal processingmodule 105 incorporating TruSurround and/or WOW is particularly usefulwhen the computer 103 is connected to the two-channel speaker system146, 147. The signal processing module 105 incorporating TruSurroundand/or WOW is also particularly useful when the speakers 146 and 147 arenot optimally placed or do not provide optimal bass response.

Circle Surround 5.1 (CS 5.1) technology, as disclosed in U.S. Pat. No.5,771,295 (the '259 patent), titled “5-2-5 MATRIX SYSTEM,” which ishereby incorporated by reference in its entirety, is adaptable for useas a multi-channel Internet audio delivery technology. CS 5.1 enablesthe matrix encoding of 5.1 high-quality channels on two channels ofaudio. These two channels can then be efficiently transmitted over theInternet using any of the popular compression schemes available (Mp3,RealAudio, WMA, etc.) and received in useable form on the client side.At the client side, in the computer 103, the CS 5.1 decoder 104 is usedto decode a full multi-channel audio output from the two channelsstreamed over the Internet. The CS 5.1 system is referred to as a 5-2-5system in the '259 patent because five channels are encoded into twochannels, and then the two channels are decoded back into five channels.The “5.1” designation, as used in “CS 5.1,” typically refers to the fivechannels (e.g., left, right, center, left-rear (also known asleft-surround), right-rear (also known as right-surround)) and anoptional subwoofer channel derived from the five channels.

Although the '259 patent describes the CS 5.1 system using hardwareterminology and diagrams, one of ordinary skill in the art willrecognize that a hardware-oriented description of signal processingsystems, even signal processing systems intended to be implemented insoftware, is common in the art, convenient, and efficiently provides aclear disclosure of the signal processing algorithms. One of ordinaryskill in the art will recognize that the CS 5.1 system described in the'259 patent can be implement in software by using digital signalprocessing algorithms that mimic the operation of the describedhardware.

Use of CS 5.1 technology to stream multi-channel audio signals creates abackwardly compatible, fully upgradeable Internet audio delivery system.For example, because the CS 5.1 decoding system 104 can create amulti-channel output from any audio source in the group 101, theoriginal format of the audio signal prior to streaming can include awide variety of encoded and non-encoded source formats including theDolby Surround source 111, the conventional stereo source 112, or themonaural source 113. This creates a seamless architecture for both thewebsite developer performing Internet audio streaming and the listener148 receiving the audio signals over the Internet. If the websitedeveloper wants an even higher quality audio experience at the clientside, the audio source can first be encoded with CS 5.1 prior tostreaming (as in the source 110). The CS 5.1 decoding system 104 canthen generate 5.1 channels of full bandwidth audio providing an optimalaudio experience.

The surround channels that are derived from the CS 5.1 decoder 104 areof higher quality as compared to other available systems. While thebandwidth of the surround channels in a Dolby ProLogic system is limitedto 7 KHz monaural, CS 5.1 provides stereo surround channels that arelimited only by the bandwidth of the transmission media.

The disclosed Internet delivery system 100 is also compatible withclient-side systems 103 that are not equipped for multi-channel audiooutput. For two-channel output (e.g., using the loudspeakers 146,147), avirtualization technology can be used to combine the multi-channel audiosignals for playback on a two-speaker system without loss of surroundsound effects. In one embodiment, “TruSurround” multi-channelvirtualization technology, as disclosed in U.S. Pat. No. 5,912,976,incorporated herein by reference in its entirety, is used on the Clientside to present the decoded surround information in a two-channel,two-speaker format. In addition, the signal processing techniquesdisclosed in U.S. Pat. Nos. 5,661,808 and 5,892,830, both of which areincorporated herein by reference, can be used on both the client andserver side to spatially enhance multi-channel, multi-speakerimplementations. In one embodiment, the WOW technology can be used inthe computer 103 or server-side to enhance the spatial and basscharacteristics of the streamed audio signal. The WOW technology, as isdisclosed herein in connection with FIGS. 4-17 and in U.S. patentapplication Ser. No. 09/411,143, titled “ACOUSTIC CORRECTION APPARATUS,”which is hereby incorporated by reference in its entirety.

Use of the Internet multi-channel audio delivery system 100 as disclosedherein solves the problem of limited bandwidth for delivering qualitysurround sound over the Internet. Moreover, the system can be deployedin a segmented fashion either at the client side, the server side, orboth, thereby reducing compatibility problems and allowing for variouslevels of sound enrichment. This combination of wide sourcecompatibility, flexible transmission requirements, high surround qualityand additional audio enhancements, such as WOW, uniquely solves theissues and problems of streaming audio over the Internet.

Due to the highly compressed nature of Internet music streams, thequality of the received audio can be very poor. Through the use of “WOW”technology, and other audio enhancement technologies, the perceivedquality of music transmitted and distributed over the Internet can besignificantly improved.

The WOW technology (as shown in FIG. 4) combines three processes: (1)psychoacoustic audio processing to create a wider soundstage, (2) anacoustic correction process to increase the perceived height and clarityof the audio image, and (3) bass enhancement processing to create theperception of low bass from the small speakers or headphones typicallyused with multi-media systems and portable audio players. The WOWcombination of technologies has been found to be uniquely suited tocompensating for the quality limitations of highly compressed audio.

Licensing and Management of the Enhancement Process

Although FIG. 1 shows WOW, and other audio enhancement technologies(e.g., CS 5.1, TruSurround) as being implemented on the client side (inthe client computer 103), these and other enhancement technologies canalso be implemented in host based (server-side signal processing)software. In one embodiment, the server-side signal processing islicensed to various Internet broadcasters to allow the broadcaster toproduce enhanced Internet audio broadcasts. Such enhanced Internet audiobroadcasts provide a significant market advantage regarding impact andquality of their transmissions. In one embodiment, the use of theserver-side enhancement software is controlled in such a way as toprovide an advantage to broadcasting partners using enhanced signalprocessing technology (e.g., WOW, TruSurround, CS 5.1, etc), whileproviding an incentive to other broadcasters to include the enhancedsignal processing technology in their broadcasts.

FIG. 2 is a block diagram showing the computer systems used by abroadcast user and a broadcast partner. The broadcast user has apersonal computer 103 (PC) system of the type ordinarily used foraccessing the Internet. The broadcast user's PC system includes hardware206, software 207 and an attached video monitor 203. The PC system 103is connected via the Internet 219 as shown, to a server system 220 usedby the broadcast partner. The broadcast partner's server 220 contains adownloadable browser interface 210, which can include enhanced signalprocessing technology audio processing capabilities (e.g., WOW,TruSurround, CS 5.1, etc.) or one of many other unique features. Uponaccessing the server 220 (e.g., by accessing an Internet website of thebroadcast partner), the user is given the option of downloading thepartner's browser interface 210 and the option of including the uniqueprocessing capabilities of the browser interface 210. In one embodiment,when the user initially accesses the web site of a broadcast partner(i.e., the server 220), the user is encouraged to download an additionalsoftware application, such as a unique enhancement technology, toenhance the audio quality of the broadcast provided by the broadcastpartner. In one embodiment, the browser interface 210 is disabled whenthe computer 103 is playing streaming audio from a non-partner server230.

In one embodiment, the browser interface 210 also includes a customizedlogo, or other message, associated with the broadcast partner. Oncedownloaded, the browser interface 210 display the customized logowhenever streaming audio broadcasts are received from the broadcastpartner's website (e.g., from the server 220). If accepted anddownloaded by the user, the enhanced browser interface 210 can alsoreside in the broadcast user's PC 103. In one embodiment, the enhancedbrowser interface 210 contacts an access server 240 to determine if theserver 220 is a partner server. In one embodiment, the access server iscontrolled by the licensor (e.g., the owner) of the audio enhancementtechnology provided by the enhanced browser interface 210. In oneembodiment, the enhanced browser interface 210 allows the listener 148to turn audio enhancement (e.g., WOW, CS 5.1, TruSurround, etc.) on andoff, and it allows the listener 148 to control the operation of theaudio enhancement.

As part of an Internet audio enhancement system, the enhanced signalprocessing technology can be used as an integral part of thebrowser-controlled user interface 210 that can be dynamically customizedby the broadcast partner. In one embodiment, the browser partnerdynamically customizes the interface 210 by accessing any user thatdownloaded the interface and is connected to the Internet. Onceaccessed, the broadcast partner can modify the customized logo or anymessage displayed by the browser interface on the user's computer.

Since the enhancement software processing capabilities can be offeredfrom many different websites as standalone application software, and insome cases can be offered for free, an incentive is used to persuadebroadcast partners to incorporate the WOW (or other) technology in theircustomized browser interfaces so that market penetration or revenuegeneration goals are achieved.

The system disclosed herein provides a method of delivering a browserinterface having audio enhancement, or other unique characteristics to auser, while still providing an incentive for additional broadcastpartners to include such unique characteristics in their browsers. Byway of example, the description that follows assumes that WOW technologyis included in the browser interface 210 delivered over the Internet toa user. However, it can be appreciated by one of ordinary skill in theart that the invention is applicable to any audio enhancementtechnology, including TruSurround, CS 5.1, or any feature for thatmatter which may be associated with an internet browser or otherdownloadable piece of software.

The incentive provided to persuade broadcast partners to offer aWOW-enabled browser is the display of the broadcast partner's customizedlogo on the browser screens of users that download the WOW-enabledbrowser interface 210 from the broadcast partner. Offering WOWtechnology to broadcast partners allows the partners to offer a uniqueaudio player interface to their users. The more users that download theWOW browser 210 from a broadcast partner, the more places the broadcastpartner's logo is displayed. Once WOW technology has been downloaded, itcan automatically display a browser-based interface, customized by thepartner. This interface can either simply provide user control of WOW orintegrate full stream access and playback controls in addition to theWOW controls.

The operation and management of the browser-based interface 210including WOW and the partner's customized logo is described inconnection with the flowchart 300 of FIG. 3. The flowchart of FIG. 3describes the operations after a user has already downloaded theWOW-enabled browser interface 210 from a broadcast partner. In FIG. 3, auser begins from a start block 320 in which a software audio playbackdevice, such as Microsoft's Media Player or the Real Player, isinitiated on the user's PC 103. In one embodiment, the control software(that implements to the flowchart in FIG. 3) resides in the WOWtechnology initialization code, which is started when an associatedmedia player is initiated by a user. After the start block 320,operational flow of the management system 300 enters a decision block322 where it is determined whether audio playback is performed throughInternet streaming or via a locally stored audio file on the user's PC103. If audio playback is from a local file (e.g., one resident on thePC's hard disk, CD, etc.) then the flowchart 300 advances to a block 324where the user is presented with a customizable local (non-browser)interface that displays the style and logo of the partner from which WOWwas previously downloaded. Alternatively, if audio playback using theWOW-based player is accomplished through data streaming (e.g., from theInternet), then the process 300 advances to a decision block 326. In thedecision block 326, the process determines whether the source of thedata stream is a WOW broadcast partner. If the source is a broadcastpartner, then control enters the state 328 where the partner'scustomized browser-based interface 210 is displayed on the user's videoscreen 203. Conversely, if the source is not a broadcast partner, thencontrol enters a state 330 in which the WOW feature resident on theuser's PC is disabled when receiving streamed data from the non-partnerbroadcast site. If the user reverts to playback of local files, thecustomized interface displaying the style and logo of the originaldownload site is displayed.

Thus, in operation, the listener 148 selects a URL that provided adesired streaming audio program. The customized browser interface 210sends the URL address to the WOW access server 240. In response, the WOWaccess server 240 sends an enable-WOW or a disable-WOW message back tothe customized browser interface 210. The WOW access server 240 sendsthe enable-WOW message if the URL corresponds to a partner server (i.e.,a WOW licensee site). The WOW access server 240 sends the disable-WOWmessage if the URL corresponds to a non-partner server (i.e., a sitethat has not licensed the WOW technology). The customized browserinterface 210 receives the enable/disable message and enables ordisables the client-side WOW processor accordingly. Again, it isemphasized that WOW is used in the above description by way of example,and that the above features can be used with other audio enhancementtechnologies including, for example, TruSurround, CS 5.1, DolbySurround, etc.

FIG. 4 is a block diagram of a WOW acoustic correction apparatus 420comprising, in series, a stereo image correction system 422, a bassenhancement system 401, and a stereo image enhancement system 424. Theimage correction system 422 provides a left stereo signal and a rightstereo signal to the bass enhancement unit 401. The bass enhancementunit outputs left and right stereo signals to respective left and rightinputs of the stereo image enhancement device 424. The stereo imageenhancement system 424 processes the signals and provides a left outputsignal 430 and a right output signal 432. The output signals 430 and 432may in turn be connected to some other form of signal conditioningsystem, or they may be connected directly to loudspeakers or headphones(not shown).

When connected to loudspeakers, the correction system 420 corrects fordeficiencies in the placement of the loudspeakers, the image created bythe loudspeakers, and the low frequency response produced by theloudspeakers. The sound correction system 420 enhances spatial andfrequency response characteristics of the sound reproduced by theloudspeakers. In the audio correction system 420, the image correctionmodule 422 corrects the listener-perceived vertical image of an apparentsound stage reproduced by the loudspeakers, the bass enhancement module401 improves the listener-perceived bass response of the sound, and theimage enhancement module 424 enhances the listener-perceived horizontalimage of the apparent sound stage.

The correction apparatus 420 improves the sound reproduced byloudspeakers by compensating for deficiencies in the sound reproductionenvironment and deficiencies of the loudspeakers. The apparatus 420improves reproduction of the original sound stage by compensating forthe location of the loudspeakers in the reproduction environment. Thesound-stage reproduction is improved in a way that enhances both thehorizontal and vertical aspects of the apparent (i.e. reproduced) soundstage over the audible frequency spectrum. The apparatus 420advantageously modifies the reverberant sounds that are easily perceivedin a live sound stage such that the reverberant sounds are alsoperceived by the listener in the reproduction environment, even thoughthe loudspeakers act as point sources with limited ability. Theapparatus 420 also compensates for the fact that microphones oftenrecord sound differently from the way the human hearing system perceivessound. The apparatus 420 uses filters and transfer functions that mimichuman hearing to correct the sounds produced by the microphone.

The sound system 420 adjusts the apparent azimuth and elevation point ofa complex sound by using the characteristics of the human auditoryresponse. The correction is used by the listener's brain to provideindications of the sound's origin. The correction apparatus 420 alsocorrects for loudspeakers that are placed at less than ideal conditions,such as loudspeakers that are not in the most acoustically-desirablelocation.

To achieve a more spatially correct response for a given sound system,the acoustic correction apparatus 420 uses certain aspects of thehead-related-transfer-functions (HRTFs) in connection with frequencyresponse shaping of the sound information to correct both the placementof the loudspeakers, to correct the apparent width and height of thesound stage, and to correct for inadequacies in the low-frequencyresponse of the loudspeakers.

Thus, the acoustic correction apparatus 420 provides a more natural andrealistic sound stage for the listener, even when the loudspeakers areplaced at less than ideal locations and when the loudspeakers themselvesare inadequate to properly reproduce the desired sounds.

The various sound corrections provided by the correction apparatus areprovided in an order such that subsequent correction does not interferewith prior corrections. In one embodiment, the corrections are providedin a desirable order such that prior corrections provided by theapparatus 420 enhance and contribute to the subsequent correctionsprovided by the apparatus 420.

In one embodiment, the correction apparatus 420 simulates a surroundsound system with improved bass response. The correction apparatus 420creates the illusion that multiple loudspeakers are placed around thelistener, and that audio information contained in multiple recordingtracks is provided to the multiple speaker arrangement.

The acoustic correction system 420 provides a sophisticated andeffective system for improving the vertical, horizontal, and spectralsound image in an imperfect reproduction environment. The imagecorrection system 422 first corrects the vertical image produced by theloudspeakers. Then the bass enhanced system 401 adjusts the lowfrequency components of the sound signal in a manner that enhances thelow frequency output of small loudspeakers that do no provide adequatelow frequency reproduction capabilities. Finally, the horizontal soundimage is corrected by the image enhancement system 424.

The vertical image enhancement provided by the image correction system422 typically includes some emphasis of the lower frequency portions ofthe sound, and thus providing vertical enhancement before the bassenhancement system 401 contributes to the overall effect of the bassenhancement processing. The bass enhancement system 401 provides somemixing of the common portions of the left and right portions of the lowfrequency information in a stereophonic signal (common-mode). Bycontrast, the horizontal image enhancement provided by the imageenhancement system 424 provides enhancement and shaping of thedifferences between the left and right portions (differential-mode) ofthe signal. Thus, in the correction system 420, bass enhancement isadvantageously provided before horizontal image enhancement in order tobalance the common-mode and differential-mode portions of thestereophonic signal to produce a pleasing effect for the listener.

As disclosed above, the stereo image correction system 422, the bassenhancement system 401, and the stereo image enhancement system 424cooperate to overcome acoustic deficiencies of a sound reproductionenvironment. The sound reproduction environments may be as large as atheater complex or as small as a portable electronic keyboard.

FIG. 5A depicts a graphical representation of a desired frequencyresponse characteristic, appearing at the outer ears of a listener,within an audio reproduction environment. The curve 560 is a function ofsound pressure level (SPL), measured in decibels, versus frequency. Ascan be seen in FIG. 5A, the sound pressure level is relatively constantfor all audible frequencies. The curve 560 can be achieved fromreproduction of pink noise through a pair of ideal loudspeakers placeddirectly in front of a listener at approximately ear level. Pink noiserefers to sound delivered over the audio frequency spectrum having equalenergy per octave. In practice, the flat frequency response of the curve560 may fluctuate in response to inherent acoustic limitations ofspeaker systems.

The curve 560 represents the sound pressure levels that exist beforeprocessing by the ear of a listener. The flat frequency responserepresented by the curve 560 is consistent with sound emanating towardsthe listener 148, when the loudspeakers are located spaced apart andgenerally in front of the listener 148. The human ear processes suchsound, as represented by the curve 560, by applying its own auditoryresponse to the sound signals. This human auditory response is dictatedby the outer pinna and the interior canal portions of the ear.

Unfortunately, the frequency response characteristics of many home andsmall computer sound reproduction systems do not provide the desiredcharacteristic shown in FIG. 5A. On the contrary, loudspeakers may beplaced in acoustically-undesirable locations to accommodate otherergonomic requirements. Sound emanating from the loudspeakers 146 and147 may be spectrally distorted by the mere placement of theloudspeakers 146 and 147 with respect to the listener 148. Moreover,objects and surfaces in the listening environment may lead toabsorption, or amplitude distortion, of the resulting sound signals.Such absorption is often prevalent among higher frequencies.

As a result of both spectral and amplitude distortion, a stereo imageperceived by the listener 148 is spatially distorted providing anundesirable listening experience. FIGS. 5B-5D graphically depict levelsof spatial distortion for various sound reproduction systems andlistening environments. The distortion characteristics depicted in FIGS.5B-5D represent sound pressure levels, measured in decibels, which arepresent near the ears of a listener.

The frequency response curve 564 of FIG. 5B has a decreasingsound-pressure level at frequencies above approximately 100 Hz. Thecurve 564 represents a possible sound pressure characteristic generatedfrom loudspeakers, containing both woofers and tweeters, which aremounted below a listener. For example, assuming the loudspeakers 146,147 contain tweeters, an audio signal played through only suchloudspeakers 146, 147 might exhibit the response of FIG. 5B.

The particular slope associated with the decreasing curve 564 varies,and may not be entirely linear, depending on the listening area, thequality of the loudspeakers, and the exact positioning of theloudspeakers within the listening area. For example, a listeningenvironment with relatively hard surfaces will be more reflective ofaudio signals, particularly at higher frequencies, than a listeningenvironment with relatively soft surfaces (e.g., cloth, carpet, acoustictile, etc). The level of spectral distortion will vary as loudspeakersare placed further from, and positioned away from, a listener.

FIG. 5C is a graphical representation of a sound-pressure versusfrequency characteristic 568 wherein a first frequency range of audiosignals are spectrally distorted, but a higher frequency range of thesignals are not distorted. The characteristic curve 568 may be achievedfrom a speaker arrangement having low to mid-frequency loudspeakersplaced below a listener and high-frequency loudspeakers positioned near,or at a listener's ear level. The sound image resulting from thecharacteristic curve 568 will have a low-frequency component positionedbelow the listener's ear level, and a high-frequency componentpositioned near the listener's ear level.

FIG. 5D is a graphical representation of a sound-pressure versusfrequency characteristic 570 having a reduced sound pressure level amonglower frequencies and an increasing sound pressure level among higherfrequencies. The characteristic 570 is achieved from a speakerarrangement having mid to low-frequency loudspeakers placed below alistener and high-frequency loudspeakers positioned above a listener. Asthe curve 570 of FIG. 4D indicates, the sound pressure level atfrequencies above 1000 Hz may be significantly higher than lowerfrequencies, creating an undesirable audio effect for a nearby listener.The sound image resulting from the characteristic curve 570 will have alow-frequency component positioned below the listener 148, and ahigh-frequency component positioned above the listener 148.

The audio characteristics of FIGS. 5B-5D represent various soundpressure levels obtainable in a common listening environment and heardby the listener. The audio response curves of FIGS. 5B-5D are but a fewexamples of how audio signals present at the ears of a listener aredistorted by various audio reproduction systems. The exact level ofspatial distortion at any given frequency will vary widely depending onthe reproduction system and the reproduction environment. The apparentlocation can be generated for a speaker system defined by apparentelevation and azimuth coordinates, with respect to a fixed listener,which are different from those of actual speaker locations.

FIG. 10 is block diagram of the stereo image correction system 422,which inputs the left and right stereo signals 426 and 428. Theimage-correction system 422 corrects the distorted spectral densities ofvarious sound systems by advantageously dividing the audible frequencyspectrum into a first frequency component, containing relatively lowerfrequencies, and a second frequency component, containing relativelyhigher frequencies. Each of the left and right signals 426 and 428 isseparately processed through corresponding low-frequency correctionsystems 1080, 1082, and high-frequency correction systems 1084 and 1086.It should be pointed out that in one embodiment the correction systems1080 and 1082 will operate in a relatively “low” frequency range ofapproximately 100 Hz to 1000 Hz, while the correction systems 1084 and1086 will operate in a relatively “high” frequency range ofapproximately 1000 Hz to 10,000 Hz. This is not to be confused with thegeneral audio terminology wherein low frequencies represent frequenciesup to 100 Hz, mid frequencies represent frequencies between 100 Hz to 4kHz, and high frequencies represent frequencies above 4 kHz.

By separating the lower and higher frequency components of the inputaudio signals, corrections in sound pressure level can be made in onefrequency range independent of the other. The correction systems 1080,1082, 1084, and 1086 modify the input signals 426 and 428 to correct forspectral and amplitude distortion of the input signals upon reproductionby loudspeakers. The resultant signals, along with the original inputsignals 426 and 428, are combined at respective summing junctions 1090and 1092. The corrected left stereo signal, L_(c), and the correctedright stereo signal, R_(c), are provided along outputs to the bassenhancement unit 401.

The corrected stereo signals provided to the bass unit 401 have a flat,i.e., uniform, frequency response appearing at the ears of the listener148. This spatially-corrected response creates an apparent source ofsound which, when played through the loudspeakers 146,147, is seeminglypositioned directly in front of the listener 148.

Once the sound source is properly positioned through energy correctionof the audio signal, the bass enhancement unit 101 corrects for lowfrequency deficiencies in the loudspeakers 146, 147 and providesbass-corrected left and right channel signals to the stereo enhancementsystem 424. The stereo enhancement system 424 conditions the stereosignals to broaden (horizontally) the stereo image emanating from theapparent sound source. As will be discussed in conjunction with FIGS. 8Aand 8B, the stereo image enhancement system 424 can be adjusted througha stereo orientation device to compensate for the actual location of thesound source.

In one embodiment, the stereo enhancement system 424 equalizes thedifference signal information present in the left and right stereosignals

The left and right signals 1094, 1096 provided from the bass enhancementunit 401 are inputted by the enhancement system 424 and provided to adifference-signal generator 1001 and a sum signal generator 1004. Adifference signal (L_(c)−R_(c)) representing the stereo content of thecorrected left and right input signals, is presented at an output 1002of the difference signal generator 1001. A sum signal, (L_(c)+R_(c))representing the sum of the corrected left and right stereo signals isgenerated at an output 1006 of the sum signal generator 1004.

The sum and difference signals at outputs 1002 and 1006 are provided tooptional level-adjusting devices 1008 and 1010, respectively. Thedevices 1008 and 1010 are typically potentiometers or similarvariable-impedance devices. Adjustment of the devices 1008 and 1010 istypically performed manually to control the base level of sum anddifference signal present in the output signals. This allows a user totailor the level and aspect of stereo enhancement according to the typeof sound reproduced, and depending on the user's personal preferences.An increase in the base level of the sum signal emphasizes the audioinformation at a center stage positioned between a pair of loudspeakers.Conversely, an increase in the base level of difference signalemphasizes the ambient sound information creating the perception of awider sound image. In some audio arrangements where the music type andsystem configuration parameters are known, or where manual adjustment isnot practical, the adjustment devices 1008 and 1010 may be eliminatedrequiring the sum and difference-signal levels to be predetermined andfixed.

The output of the device 1010 is fed into a stereo enhancement equalizer1020 at an input 1022. The equalizer 1020 spectrally shapes thedifference signal appearing at the input 1022.

The shaped difference signal 1040 is provided to a mixer 1042, whichalso receives the sum signal from the device 1008. In one embodiment,the stereo signals 1094 and 1096 are also provided to the mixer 1042.All of these signals are combined within the mixer 1042 to produce anenhanced and spatially-corrected left output signal 1030 and rightoutput signal 1032.

Although the input signals 426 and 428 typically represent correctedstereo source signals, they may also be synthetically generated from amonophonic source.

FIGS. 6A-6C are graphical representations of the levels of spatialcorrection provided by “low” and “high”-frequency correction systems1080, 1082, 1084, 1086 in order to obtain a relocated image generatedfrom a pair of stereo signals.

Referring initially to FIG. 6A, possible levels of spatial correctionprovided by the correction systems 1080 and 1082 are depicted as curveshaving different amplitude-versus-frequency characteristics. The maximumlevel of correction, or boost (measured in dB), provided by the systems1080 and 1082 is represented by a correction curve 650. The curve 650provides an increasing level of boost within a first frequency range ofapproximately 100 Hz and 1000 Hz. At frequencies above 1000 Hz, thelevel of boost is maintained at a fairly constant level. A curve 652represents a near-zero level of correction.

To those skilled in the art, a typical filter is usually characterizedby a pass-band and stop-band of frequencies separated by a cutofffrequency. The correction curves, of FIGS. 6A-6C, althoughrepresentative of typical signal filters, can be characterized by apass-band, a stop-band, and a transition band. A filter constructed inaccordance with the characteristics of FIG. 6A has a pass-band aboveapproximately 1000 Hz, a transition-band between approximately 100 and1000 Hz, and a stop-band below approximately 100 Hz. Filters accordingto FIG. 6B have pass-bands above approximately 10 kHz, transition-bandsbetween approximately 1 kHz and 10 kHz, and a stop-band belowapproximately 1 kHz. Filters according to FIG. 6C have a stop-band aboveapproximately 10 kHz, transition-bands between approximately 1 kHz and10 kHz, and pass-bands below approximately 1 kHz. In one embodiment, thefilters are first-order filters.

As can be seen in FIGS. 6A-6C, spatial correction of an audio signal bythe systems 1080, 1082, 1084, and 1086 is substantially uniform withinthe pass-bands, but is largely frequency-dependent within the transitionbands. The amount of acoustic correction applied to an audio signal canbe varied as a function of frequency through adjustment of the stereoimage correction system, which varies the slope of the transition bandsof FIGS. 6A-6C. As a result, frequency-dependent correction is appliedto a first frequency range between 100 Hz and 1000 Hz, and applied to asecond frequency range of 1000 Hz to 10,000 Hz. An infinite number ofcorrection curves are possible through independent adjustment of thecorrection systems 1080, 1082, 1084 and 1086.

In accordance with one embodiment, spatial correction of the higherfrequency stereo-signal components occurs between approximately 1000 Hzand 10,000 Hz. Energy correction of these signal components may bepositive, i.e., boosted, as depicted in FIG. 6B, or negative, i.e.,attenuated, as depicted in FIG. 6C. The range of boost provided by thecorrection systems 1084, 1086 is characterized by a maximum-boost curve660 and a minimum-boost curve 662. Curves 664, 666, and 668 representstill other levels of boost, which may be required to spatially correctsound emanating from different sound reproduction systems. FIG. 6Cdepicts energy-correction curves that are essentially the inverse ofthose in FIG. 6B.

Since the lower frequency and higher frequency correction factors,represented by the curves of FIGS. 6A-6C, are added together, there is awide range of possible spatial correction curves applicable between thefrequencies of 100 to 10,000 Hz. FIG. 6D is a graphical representationdepicting a range of composite spatial correction characteristicsprovided by the stereo image correction system 422. Specifically, thesolid line curve 680 represents a maximum level of spatial correctioncomprised of the curve 650 (shown in FIG. 6A) and the curve 660 (shownin FIG. 6B). Correction of the lower frequencies may vary from the solidcurve 680 through the range designated by θ₁. Similarly, correction ofthe higher frequencies may vary from the solid curve 680 through therange designated by θ₂. Accordingly, the amount of boost applied to thefirst frequency range of 100 Hz to 1000 Hz varies between approximately0 and 15 dB, while the correction applied to the second frequency rangeof 1000 to 10,000 Hertz may vary from approximately 15 dB to 30 dB.

Turning now to the stereo image enhancement aspect of the presentinvention, a series of perspective-enhancement, or normalization curves,is graphically represented in FIG. 7. The signal (L_(c)−R_(c))_(p)represents the processed difference signal, which has been spectrallyshaped according to the frequency-response characteristics of FIG. 7.These frequency-response characteristics are applied by the equalizer1020 depicted in FIG. 10 and are partially based upon HRTF principles.

In general, selective amplification of the difference signal enhancesany ambient or reverberant sound effects which may be present in thedifference signal but which are masked by more intense direct-fieldsounds. These ambient sounds are readily perceived in a live sound stageat the appropriate level. In a recorded performance, however, theambient sounds are attenuated relative to a live performance. Byboosting the level of difference signal derived from a pair of stereoleft and right signals, a projected sound image can be broadenedsignificantly when the image emanates from a pair of loudspeakers placedin front of a listener.

The perspective curves 790, 792, 794, 796, and 798 of FIG. 7 aredisplayed as a function of gain against audible frequencies displayed inlog format. The different levels of equalization between the curves ofFIG. 7 are required to account for various audio reproduction systems.In one embodiment, the level of difference-signal equalization is afunction of the actual placement of loudspeakers relative to a listenerwithin an audio reproduction system. The curves 790, 792, 794, 796, and798 generally display a frequency contouring characteristic whereinlower and higher difference-signal frequencies are boosted relative to amid-band of frequencies.

According to one embodiment, the range for the perspective curves ofFIG. 7 is defined by a maximum gain of approximately 10-15 dB located atapproximately 125 to 150 Hz. The maximum gain values denote a turningpoint for the curves of FIG. 7 whereby the slopes of the curves 790,792, 794, 796, and 798 change from a positive value to a negative value.Such turning points are labeled as points A, B, C, D, and E in FIG. 7.The gain of the perspective curves decreases below 125 Hz at a rate ofapproximately 6 dB per octave. Above 125 Hz, the gain of the curves ofFIG. 7 also decreases, but at variable rates, towards a minimum-gainturning point of approximately −2 to +10 dB. The minimum-gain turningpoints vary significantly between the curves 790, 792, 794, 796, and798. The minimum-gain turning points are labeled as points A′, B′, C′,D′, and E′, respectively. The frequencies at which the minimum-gainturning points occur varies from approximately 2.1 kHz for curve 790 toapproximately 5 kHz for curve 798. The gain of the curves 790, 792, 794,796, and 798 increases above their respective minimum-gain frequenciesup to approximately 10 kHz. Above 10 kHz, the gain applied by theperspective curves begins to level off. An increase in gain willcontinue to be applied by all of the curves, however, up toapproximately 20 kHz, i.e., approximately the highest frequency audibleto the human ear.

The preceding gain and frequency figures are merely design objectivesand the actual figures will likely vary from system to system. Moreover,adjustment of the signal level devices 1008 and 1010 will affect themaximum and minimum gain values, as well as the gain separation betweenthe maximum-gain frequency and the minimum-gain frequency.

Equalization of the difference signal in accordance with the curves ofFIG. 7 is intended to boost the difference signal components ofstatistically lower intensity without overemphasizing thehigher-intensity difference signal components. The higher-intensitydifference signal components of a typical stereo signal are found in amid-range of frequencies between approximately 1 kHz to 4 kHz. The humanear has a heightened sensitivity to this same mid-range of frequencies.Accordingly, the enhanced left and right output signals 1030 and 1032produce a much improved audio effect because ambient sounds areselectively emphasized to fully encompass a listener within a reproducedsound stage.

As can be seen in FIG. 7, difference signal frequencies below 125 Hzreceive a decreased amount of boost, if any, through the application ofthe perspective curve. This decrease is intended to avoidover-amplification of very low, i.e., bass, frequencies. With many audioreproduction systems, amplifying an audio difference signal in thislow-frequency range can create an unpleasurable and unrealistic soundimage having too much bass response. Examples of such audio reproductionsystems include near-field or low-power audio systems, such asmultimedia computer systems, as well as home stereo systems. A largedraw of power in these systems may cause amplifier “clipping” duringperiods of high boost, or it may damage components of the audio systemincluding the loudspeakers. Limiting the bass response of the differencesignal also helps avoid these problems in most near-field audioenhancement applications.

In accordance with one embodiment, the level of difference signalequalization in an audio environment having a stationary listener isdependent upon the actual speaker types and their locations with respectto the listener. The acoustic principles underlying this determinationcan best be described in conjunction with FIGS. 8A and 8B. FIGS. 8A and8B are intended to show such acoustic principles with respect to changesin azimuth of a speaker system.

FIG. 8A depicts a top view of a sound reproduction environment havingloudspeakers 800 and 802 placed slightly forward of, and pointedtowards, the sides of a listener 804. The loudspeakers 800 and 802 arealso placed below the listener 804 at a elevational position similar tothat of the loudspeakers 146, 147 shown in FIG. 2. Reference planes Aand B are aligned with ears 806, 808 of the listener 804. The planes Aand B are parallel to the listener's line-of-sight as shown.

The location of the loudspeakers preferably correspond to the locationsof the loudspeakers 810 and 812. In one embodiment, when theloudspeakers cannot be located in a desired position, enhancement of theapparent sound image can be accomplished by selectively equalizing thedifference signal, i.e., the gain of the difference signal will varywith frequency. The curve 790 of FIG. 7 represents the desired level ofdifference-signal equalization with actual speaker locationscorresponding to the phantom loudspeakers 810 and 812.

The present invention also provides a method and system for enhancingaudio signals. The sound enhancement system improves the realism ofsound with a unique sound enhancement process. Generally speaking, thesound enhancement process receives two input signals, a left inputsignal and a right input signal, and in turn, generates two enhancedoutput signals, a left output signal and a right output signal.

The left and right input signals are processed collectively to provide apair of left and right output signals. In particular, the enhancedsystem embodiment equalizes the differences that exist between the twoinput signals in a manner, which broadens and enhances the perceivedbandwidth of the sounds. In addition, many embodiments adjust the levelof the sound that is common to both input signals so as to reduceclipping.

Although the embodiments are described herein with reference to onesound enhancement systems, the invention is not so limited, and can beused in a variety of other contexts in which it is desirable to adaptdifferent embodiments of the sound enhancement system to differentsituations.

A typical small loudspeaker system used for multimedia computers,automobiles, small stereophonic systems, portable stereophonic systems,headphones, and the like, will have an acoustic output response thatrolls off at about 150 Hz. FIG. 9 shows a curve 906 correspondingapproximately to the frequency response of the human ear. FIG. 9 alsoshows the measured response 908 of a typical small computer loudspeakersystem that uses a high-frequency driver (tweeter) to reproduce the highfrequencies, and a four-inch midrange-bass driver (woofer) to reproducethe midrange and bass frequencies. Such a system employing two driversis often called a two-way system. Loudspeaker systems employing morethan two drivers are known in the art and will work with the presentinvention. Loudspeaker systems with a single driver are also known andwill work with the present invention. The response 908 is plotted on arectangular plot with an X-axis showing frequencies from 20 Hz to 20kHz. This frequency band corresponds to the range of normal humanhearing. The Y-axis in FIG. 9 shows normalized amplitude response from 0dB to −50 dB. The curve 908 is relatively flat in a midrange frequencyband from approximately 2 kHz to 10 kHz, showing some roll off above 10kHz. In the low frequency ranges, the curve 908 exhibits a low-frequencyroll off that begins in a midbass band between approximately 150 Hz and2 kHz such that below 150 Hz, the loudspeaker system produces verylittle acoustic output.

The location of the frequency bands shown in FIG. 9 are used by way ofexample and not by way of limitation. The actual frequency ranges of thedeep bass band, midbass band, and midrange band vary according to theloudspeaker and the application for which the loudspeaker is used. Theterm deep bass is used, generally, to refer to frequencies in a bandwhere the loudspeaker produces an output that is less accurate ascompared to the loudspeaker output at higher frequencies, such as, forexample, in the midbass band. The term midbass band is used, generally,to refer to frequencies above the deep bass band. The term midrange isused, generally, to refer to frequencies above the midbass band.

Many cone-type drivers are very inefficient when producing acousticenergy at low frequencies where the diameter of the cone is less thanthe wavelength of the acoustic sound wave. When the cone diameter issmaller than the wavelength, maintaining a uniform sound pressure levelof acoustic output from the cone requires that the cone excursion beincreased by a factor of four for each octave (factor of 2) that thefrequency drops. The maximum allowable cone excursion of the driver isquickly reached if one attempts to improve low-frequency response bysimply boosting the electrical power supplied to the driver.

Thus, the low-frequency output of a driver cannot be increased beyond acertain limit, and this explains the poor low-frequency sound quality ofmost small loudspeaker systems. The curve 908 is typical of most smallloudspeaker systems that employ a low-frequency driver of approximatelyfour inches in diameter. Loudspeaker systems with larger drivers willtend to produce appreciable acoustic output down to frequencies somewhatlower than those shown in the curve 908, and systems with smallerlow-frequency drivers will typically not produce output as low as thatshown in the curve 908.

As discussed above, to date, a system designer has had little choicewhen designing loudspeaker systems with extended low-frequency response.Previously known solutions were expensive and produced loudspeakers thatwere too large for the desktop. One popular solution to thelow-frequency problem is the use of a sub-woofer, which is usuallyplaced on the floor near the computer system. Sub-woofers can provideadequate low-frequency output, but they are expensive, and thusrelatively uncommon as compared to inexpensive desktop loudspeakers.

Rather than use drivers with large diameter cones, or a sub-woofer, anembodiment of the present invention overcomes the low-frequencylimitations of small systems by using characteristics of the humanhearing system to produce the perception of low-frequency acousticenergy, even when such energy is not produced by the loudspeaker system.

In one embodiment, the bass enhancement processor 401 uses a bass punchunit 1120, shown in FIG. 11. In one embodiment, the bass punch unit 1120uses an Automatic Gain Control (AGC) comprising a linear amplifier withan internal servo feedback loop. The servo automatically adjusts theaverage amplitude of the output signal to match the average amplitude ofa signal on the control input. The average amplitude of the controlinput is typically obtained by detecting the envelope of the controlsignal. The control signal may also be obtained by other methods,including, for example, low pass filtering, bandpass filtering, peakdetection, RMS averaging, mean value averaging, etc.

In response to an increase in the amplitude of the envelope of thesignal provided to the input of the bass punch unit 1120, the servo loopincreases the forward gain of the bass punch unit 1120. Conversely, inresponse to a decrease in the amplitude of the envelope of the signalprovided to the input of the bass punch unit 1120, the servo loopdecreases the forward gain of the bass punch unit 1120. In oneembodiment, the gain of the bass punch unit 1120 increases more rapidlythat the gain decreases. FIG. 11 is a time domain plot that illustratesthe gain of the bass punch unit 1120 in response to a unit step input.One skilled in the art will recognize that FIG. 11 is a plot of gain asa function of time, rather than an output signal as a function of time.Most amplifiers have a gain that is fixed, so gain is rarely plotted.However, the Automatic Gain Control (AGC) in the bass punch unit 1120varies the gain of the bass punch unit 1120 in response to the envelopeof the input signal.

The unit step input is plotted as a curve 1109 and the gain is plottedas a curve 1102. In response to the leading edge of the input pulse1109, the gain rises during a period 1104 corresponding to an attacktime constant. At the end of the time period 1104, the gain 1102 reachesa steady-state gain of A₀. In response to the trailing edge of the inputpulse 1109, the gain falls back to zero during a period corresponding toa decay time constant 1106.

The attack time constant 1104 and the decay time constant 1106 aredesirably selected to provide enhancement of the bass frequencieswithout overdriving other components of the system such as the amplifierand loudspeakers. FIG. 12 is a time-domain plot 1200 of a typical bassnote played by a musical instrument such as a bass guitar, bass drum,synthesizer, etc. The plot 1200 shows a higher-frequency portion 1244that is amplitude modulated by a lower-frequency portion having amodulation envelope 1242. The envelope 1242 has an attack portion 1246,followed by a decay portion 1247, followed by a sustain portion 1248,and finally, followed by a release portion 1249. The largest amplitudeof the plot 1200 is at a peak 1250, which occurs at the point in timebetween the attack portion 1246 and the decay portion 1247.

As stated, the waveform 1244 is typical of many, if not most, musicalinstruments. For example, a guitar string, when pulled and released,will initially make a few large amplitude vibrations, and then settledown into a more or less steady state vibration that slowly decays overa long period. The initial large excursion vibrations of the guitarstring correspond to the attack portion 1246 and the decay portion 1247.The slowly decaying vibrations correspond to the sustain portion 1248and the release portions 1249. Piano strings operate in a similarfashion when struck by a hammer attached to a piano key.

Piano strings may have a more pronounced transition from the sustainportion 1248 to the release portion 1249, because the hammer does notreturn to rest on the string until the piano key is released. While thepiano key is held down, during the sustain period 1248, the stringvibrates freely with relatively little attenuation. When the key isreleased, the felt covered hammer comes to rest on the key and rapidlydamps out the vibration of the string during the release period 1249.

Similarly, a drumhead, when struck, will produce an initial set of largeexcursion vibrations corresponding to the attack portion 1246 and thedecay portion 1247. After the large excursion vibrations have died down(corresponding to the end of the decay portion 1247) the drumhead willcontinue to vibrate for a period of time corresponding to the sustainportion 1248 and release portion 1249. Many musical instrument soundscan be created merely by controlling the length of the periods1246-1249.

As described in connection with FIG. 12, the amplitude of thehigher-frequency signal is modulated by a lower-frequency tone (theenvelope), and thus, the amplitude of the higher-frequency signal variesaccording to the frequency of the lower frequency tone. Thenon-linearity of the ear will partially demodulate the signal such thatthe ear will detect the low-frequency envelope of the higher-frequencysignal, and thus produce the perception of the low-frequency tone, eventhough no actual acoustic energy was produced at the lower frequency.The detector effect can be enhanced by proper signal processing of thesignals in the midbass frequency range, typically between 100 Hz-150 Hzon the low end of the range and 150 Hz-500 Hz on the high end of therange. By using the proper signal processing, it is possible to design asound enhancement system that produces the perception of low-frequencyacoustic energy, even when using loudspeakers that are incapable ofproducing such energy.

The perception of the actual frequencies present in the acoustic energyproduced by the loudspeaker may be deemed a first order effect. Theperception of additional harmonics not present in the actual acousticfrequencies, whether such harmonics are produced by intermodulationdistortion or detection may be deemed a second order effect.

However, if the amplitude of the peak 1250 is too high, the loudspeakers(and possibly the power amplifier) will be overdriven. Overdriving theloudspeakers will cause a considerable distortion and may damage theloudspeakers.

The bass punch unit 1120 desirably provides enhanced bass in the midbassregion while reducing the overdrive effects of the peak 1250. The attacktime constant 1104 provided by the bass punch unit 1120 limits the risetime of the gain through the bass punch unit 1120. The attack timeconstant of the bass punch unit 1120 has relatively less effect on awaveform with a long attack period 1246 (slow envelope rise time) andrelatively more effect on a waveform with a short attack period 1246(fast envelope rise time).

An attack portion of a note played by a bass instrument (e.g., a bassguitar) will often begin with an initial pulse of relatively highamplitude. This peak may, in some cases, overdrive the amplifier orloudspeaker causing distorted sound and possibly damaging theloudspeaker or amplifier. The bass enhancement processor provides aflattening of the peaks in the bass signal while increasing the energyin the bass signal, thereby increasing the overall perception of bass.

The energy in a signal is a function of the amplitude of the signal andthe duration of the signal. Stated differently, the energy isproportional to the area under the envelope of the signal. Although theinitial pulse of a bass note may have a relatively large amplitude, thepulse often contains little energy because it is of short duration.Thus, the initial pulse, having little energy, often does not contributesignificantly to the perception of bass. Accordingly, the initial pulsecan usually be reduced in amplitude without significantly affecting theperception of bass.

FIG. 13 is a signal processing block diagram of the bass enhancementsystem 401 that provides bass enhancement using a peak compressor tocontrol the amplitude of pulses, such as the initial pulse, bass notes.In the system 401, a peak compressor 1302 is interposed between thecombiner 1318 and the punch unit 1120. The output of the combiner 1318is provided to an input of the peak compressor 1302, and an output ofthe peak compressor 1302 is provided to the input of the bass punch unit1120.

The peak compression unit 1302 “flattens” the envelope of the signalprovided at its input. For input signals with a large amplitude, theapparent gain of the compression unit 1302 is reduced. For input signalswith a small amplitude, the apparent gain of the compression unit 1302is increased. Thus, the compression unit reduces the peaks of theenvelope of the input signal (and fills in the troughs in the envelopeof the input signal). Regardless of the signal provided at the input ofthe compression unit 1302, the envelope (e.g., the average amplitude) ofthe output signal from the compression unit 1302 has a relativelyuniform amplitude.

FIG. 14 is a time-domain plot showing the effect of the peak compressoron an envelope with an initial pulse of relatively high amplitude. FIG.14 shows a time-domain plot of an input envelope 1414 having an initiallarge amplitude pulse followed by a longer period of lower amplitudesignal. An output envelope 1416 shows the effect of the bass punch unit1120 on the input envelope 1414 (without the peak compressor 1302). Anoutput envelope 1417 shows the effect of passing the input signal 1414through both the peak compressor 1302 and the punch unit 1120.

As shown in FIG. 14, assuming the amplitude of the input signal 1414 issufficient to overdrive the amplifier or loudspeaker, the bass punchunit does not limit the maximum amplitude of the input signal 1414 andthus the output signal 1416 is also sufficient to overdrive theamplifier or loudspeaker.

The pulse compression unit 1302 used in connection with the signal 1417,however, compresses (reduces the amplitude of) large amplitude pulses.The compression unit 1302 detects the large amplitude excursion of theinput signal 1414 and compresses (reduces) the maximum amplitude so thatthe output signal 1417 is less likely to overdrive the amplifier orloudspeaker.

Since the compression unit 1302 reduces the maximum amplitude of thesignal, it is possible to increase the gain provided by the punch unit1120 without significantly reducing the probability that the outputsignal 1417 will overdrive the amplifier or loudspeaker. The signal 1417corresponds to an embodiment where the gain of the bass punch unit 1120has been increased. Thus, during the long decay portion, the signal 1417has a larger amplitude than the curve 1416.

As described above, the energy in the signals 1414, 1416, and 1417 isproportional to the area under the curve representing each signal. Thesignal 1417 has more energy because, even though it has a smallermaximum amplitude, there is more area under the curve representing thesignal 1417 than either of the signals 1414 or 1416. Since the signal1417 contains more energy, a listener will perceive more bass in thesignal 1417.

Thus, the use of the peak compressor in combination with the bass punchunit 1120 allows the bass enhancement system to provide more energy inthe bass signal, while reducing the likelihood that the enhanced basssignal will overdrive the amplifier or loudspeaker.

The present invention also provides a method and system that improvesthe realism of sound (especially the horizontal aspects of the soundstage) with a unique differential perspective correction system.Generally speaking, the differential perspective correction apparatusreceives two input signals, a left input signal and a right inputsignal, and in turn, generates two enhanced output signals, a leftoutput signal and a right output signal as shown in connection with FIG.10.

The left and right input signals are processed collectively to provide apair of spatially corrected left and right output signals. Inparticular, one embodiment equalizes the differences, which existbetween the two input signals in a manner, which broadens and enhancesthe sound perceived by the listener. In addition, one embodiment adjuststhe level of the sound, which is common to both input signals so as toreduce clipping. Advantageously, one embodiment achieves soundenhancement with a simplified, low-cost, and easy-to-manufacturecircuit, which does not require separate circuits to process the commonand differential signals as shown in FIG. 10.

Although some embodiments are described herein with reference to varioussound enhancement system, the invention is not so limited, and can beused in a variety of other contexts in which it is desirable to adaptdifferent embodiments of the sound enhancement system to differentsituations.

FIG. 15 is a block diagram 1500 of a differential perspective correctionapparatus 1502 from a first input signal 1510 and a second input signal1512. In one embodiment the first and second input signals 1510 and 1512are stereo signals; however, the first and second input signals 1510 and1512 need not be stereo signals and can include a wide range of audiosignals. As explained in more detail below, the differential perspectivecorrection apparatus 1502 modifies the audio sound information, which iscommon to both the first and second input signals 1510 and 1512 in adifferent manner than the audio sound information, which is not commonto both the first and second input signals 1510 and 1512.

The audio information which is common to both the first and second inputsignals 1510 and 1512 is referred to as the common-mode information, orthe common-mode signal (not shown). In one embodiment, the common-modesignal does not exist as a discrete signal. Accordingly, the termcommon-mode signal is used throughout this detailed description toconceptually refer to the audio information, which exists in both thefirst and second input signals 1510 and 1512 at any instant in time.

The adjustment of the common-mode signal is shown conceptually in thecommon-mode behavior block 1520. The common-mode behavior block 1520represents the alteration of the common-mode signal. One embodimentreduces the amplitude of the frequencies in the common-mode signal inorder to reduce the clipping, which may result from high-amplitude inputsignals.

In contrast, the audio information which is not common to both the firstand second input signals 1510 and 1512 is referred to as thedifferential information or the differential signal (not shown). In oneembodiment, the differential signal is not a discrete signal, ratherthroughout this detailed description, the differential signal refers tothe audio information which represents the difference between the firstand second input signals 1510 and 1512.

The modification of the differential signal is shown conceptually in thedifferential-mode behavior block 1522. As discussed in more detailbelow, the differential perspective correction apparatus 1502 equalizesselected frequency bands in the differential signal. That is, oneembodiment equalizes the audio information in the differential signal ina different manner than the audio information in the common-mode signal.

Furthermore, while the common-mode behavior block 1520 and thedifferential-mode behavior block 1522 are represented conceptually asseparate blocks, one embodiment performs these functions with a single,uniquely adapted system. Thus, one embodiment processes both thecommon-mode and differential audio information simultaneously.Advantageously, one embodiment does not require the complicatedcircuitry to separate the audio input signals into discrete common-modeand differential signals. In addition, one embodiment does not require amixer which then recombines the processed common-mode signals and theprocessed differential signals to generate a set of enhanced outputsignals.

FIG. 16 is an amplitude-versus-frequency chart, which illustrates thecommon-mode gain at both the left and right output terminals 1530 and1532. The common-mode gain is represented with a first common-mode gaincurve 1600. As shown in the common-mode gain curve 1600, the frequenciesbelow approximately 130 hertz (Hz) are de-emphasized more than thefrequencies above approximately 130 Hz.

FIG. 17 illustrates the overall correction curve 1700 generated by thecombination of the first and second cross-over networks 1520, and 1522.The approximate relative gain values of the various frequencies withinthe overall correction curve 1700 can be measured against a zero (0) dBreference.

With such a reference, the overall correction curve 1700 shows twoturning points labeled as point A and point B. At point A, which in oneembodiment is approximately 170 Hz, the slope of the correction curvechanges from a positive value to a negative value. At point B, which inone embodiment is approximately 2 kHz, the slope of the correction curvechanges from a negative value to a positive value.

Thus, the frequencies below approximately 170 Hz are de-emphasizedrelative to the frequencies near 170 Hz. In particular, below 170 Hz,the gain of the overall correction curve 1700 decreases at a rate ofapproximately 6 dB per octave. This de-emphasis of signal frequenciesbelow 170 Hz prevents the over-emphasis of very low, (i.e. bass)frequencies. With many audio reproduction systems, over emphasizingaudio signals in this low-frequency range relative to the higherfrequencies can create an unpleasurable and unrealistic sound imagehaving too much bass response. Furthermore, over emphasizing thesefrequencies may damage a variety of audio components including theloudspeakers.

Between point A and point B, the slope of one overall correction curveis negative. That is, the frequencies between approximately 170 Hz andapproximately 2 kHz are de-emphasized relative to the frequencies near170 Hz. Thus, the gain associated with the frequencies between point Aand point B decrease at variable rates towards the maximum-equalizationpoint of −8 dB at approximately 2 kHz.

Above 2 kHz the gain increases, at variable rates, up to approximately20 kHz, i.e., approximately the highest frequency audible to the humanear. That is, the frequencies above approximately 2 kHz are emphasizedrelative to the frequencies near 2 kHz. Thus, the gain associated withthe frequencies above point B increases at variable rates towards 20kHz.

These relative gain and frequency values are merely design objectivesand the actual figures will likely vary from system to system.Furthermore, the gain and frequency values may be varied based on thetype of sound or upon user preferences without departing from the spiritof the invention. For example, varying the number of the cross-overnetworks and varying the resister and capacitor values within eachcross-over network allows the overall perspective correction curve 1700be tailored to the type of sound reproduced.

The selective equalization of the differential signal enhances ambientor reverberant sound effects present in the differential signal. Asdiscussed above, the frequencies in the differential signal are readilyperceived in a live sound stage at the appropriate level. Unfortunately,in the playback of a recorded performance the sound image does notprovide the same 360-degree effect of a live performance. However, byequalizing the frequencies of the differential signal with thedifferential perspective correction apparatus 1502, a projected soundimage can be broadened significantly so as to reproduce the liveperformance experience with a pair of loudspeakers placed in front ofthe listener.

Equalization of the differential signal in accordance with the overallcorrection curve 1700 de-emphasizes the signal components ofstatistically lower intensity relative to the higher-intensity signalcomponents. The higher-intensity differential signal components of atypical audio signal are found in a mid-range of frequencies betweenapproximately 2 kHz to 4 kHz. In this range of frequencies, the humanear has a heightened sensitivity. Accordingly, the enhanced left andright output signals produce a much improved audio effect.

The number of cross-over networks and the components within thecross-over networks can be varied in other embodiments to simulate whatare called head related transfer functions (HRTF). Head related transferfunctions describe different signal equalizing techniques for adjustingthe sound produced by a pair of loudspeakers so as to account for thetime it takes for the sound to be perceived by the left and right ears.Advantageously, an immersive sound effect can be positioned by applyingHRTF-based transfer functions to the differential signal so as to createa fully immersive positional sound field.

Examples of HRTF transfer functions which can be used to achieve acertain perceived azimuth are described in the article by E. A. B. Shawentitled “Transformation of Sound Pressure Level From the Free Field tothe Eardrum in the Horizontal Plane”, J. Acoust. Soc. Am., Vol. 106, No.6, December 1974, and in the article by S. Mehrgardt and V. Mellertentitled “Transformation Characteristics of the External Human Ear”, J.Acoust. Soc. Am., Vol. 61, No. 6, June 1977, both of which areincorporated herein by reference as though fully set forth.

In addition to music, Internet Audio is extensively utilized fortransmission of voice. Often times, voice is even more aggressivelycompressed than music resulting in poor reproduced voice quality. Bycombining voice processing technologies, such as VIP as disclosed inU.S. Pat. No. 5,459,813, and incorporated herein by reference, andTruBass, an enhancement to voice can be obtained, called “WOWVoice”,that is similar to the enhancement to music provided by WOW. As withWOW, “WOWVoice” can be implemented as a client-side technology that isinstalled in the user's computer. Exactly the same means for licensingand control discussed above can be directly applied to WOWVoice.

WOWVoice can be optimized for various applications to maximize theperceived enhancement with various bit rates and sample rates. In oneembodiment, WOWVoice includes means to restore the full frequencyspectrum to voice signals from a source that has a limited frequencyresponse. In one embodiment, WOWVoice can also combine a synthesizedMono to 3D process to create a more natural voice ambiance.

One skilled in the art will recognize that these features, and thus thescope of the present invention, should be interpreted in light of thefollowing claims and any equivalents thereto.

1. A method of enhancing an audio signal delivered over a network, themethod comprising: receiving streaming audio; obtaining at least twochannels of receive audio from the streaming audio; and enhancing the atleast two channels of receive audio for playback with one or moreprocessors, said enhancing comprising performing one or more of thefollowing: applying a virtualization enhancement to spatially enhancethe at least two channels of receive audio to thereby providesurround-sound effects in the at least two channels of receive audio,and applying a sound stage enhancement to the at least two channels ofreceive audio, said applying the sound stage enhancement comprising:enhancing a listener-perceived height of an apparent sound stageassociated with the at least two channels of receive audio to produceheight-enhanced audio, enhancing a bass response associated with the atleast two channels of receive audio to produce bass-enhanced audio, andenhancing a listener-perceived width of the apparent sound stage of theat least two channels of receive audio by at least equalizing adifference signal present in the at least two channels of receive audio.2. The method of claim 1, further comprising decoding the at least twochannels of receive audio into a 5.1 channel audio output signal.
 3. Themethod of claim 2, wherein that said enhancing the at least two channelsof receive audio comprises enhancing the 5.1. channel audio outputsignal.
 4. The method of claim 2, wherein said decoding comprisesdecoding using a Circle Surround 5.1 decoder.
 5. An apparatus forenhancing an audio signal received over a network, the apparatuscomprising: a receiver configured to receive streaming audio; aconverter configured to convert the streaming audio into at least twochannels of receive audio; a sound enhancement system comprising one ormore processors, the sound enhancement system configured to enhance theat least two channels of receive audio for playback, the soundenhancement system comprising one or more of the following: avirtualization module configured to spatially enhance the at least twochannels of receive audio to thereby provide surround-sound effects inthe at least two channels of receive audio, and a sound stageenhancement module comprising: an image enhancement module configured toenhance a listener-perceived height of an apparent sound stageassociated with the at least two channels of receive audio to produceheight-enhanced audio; a bass enhancement module configured to enhance abass response associated with the at least two channels of receive audioto produce bass-enhanced audio; and an image enhancement moduleconfigured to enhance a listener-perceived width of the apparent soundstage of the at least two channels of receive audio by at leastequalizing a difference signal present in the at least two channels ofreceive audio.
 6. The apparatus of claim 5, further comprising a decoderconfigured to decode the at least two channels of receive audio into a5.1 channel audio output signal.
 7. The apparatus of claim 6, whereinthe sound enhancement system is further configured to enhancing the atleast two channels of receive audio by at least enhancing the 5.1.channel audio output signal.
 8. The apparatus of claim 5, wherein thedecoder comprises a Circle Surround 5.1 decoder.